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Many nations within the Organization for Economic 
Cooperation and Development (OECD) have student achieve- 
ment profiles that are essentially socioeconomically and ethnically 
stratified (OECD, 2001). Increasingly, attempts to change these 
profiles have focused on reforming schools and reeducating the 
teachers within them because teachers exercise the greatest sys- 
tems influence on student achievement (Alton-Lee, 2003; Nye, 
Konstantopoulos, & Hedges, 2004; Rowan, Correnti, & Millar, 
2002). Many of these efforts, however, have met with relatively 
small and typically unrehable achievement gains, whether teach- 
ers are given prescriptions with which to work (Borman, 2005; 
Datnow et ah, 2003) or the time and resources to develop their 
own solutions (Lipman, 1997; Saxe, Gearhart, 6cNasir,2001). In 
this paper, we describe a project that impacted student outcomes 
substantively, particularly for students who scored in the bottom 
20% of achievement tests, through an approach underpinned by 
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A national literacy professional development project reduced the 
achievement gap for students experiencing difficulties in reading or writ- 
ing in 91 of New Zealand's schools. It was based on two premises: 
coherence within and between the multiple levels of the schooling and 
educational administration systems, and a focus on evidence-informed 
inquiry into effectiveness at each level of the system. Over the 2 years 
of the project's operation, these two premises interacted in ways that 
led to ongoing problem identification. Examples include how students 
understood their learning, how teachers and school leaders taught 
these students, how professional development facilitators changed their 
approaches, and how the project leaders and policy makers devel- 
oped new systems for learning. Solutions were actively and collabora- 
tively sought at all levels. Research data included assessment of student 
literacy in reading or writing; participant observations of the project 
leadership operations; interviews with principals, literacy leaders, and 
teachers; scenario responses; and interviews with facilitators. An analy- 
sis of facilitator practice early in the project illustrates how project lead- 
ership responded to the problems identified. Ongoing learning resulted 
from the interaction between facilitator feedback to teachers and their 
reflective responses to the observations of that feedback. Important to 
the success of this program is the continued feedback, not only to teach- 
ers implementing change, but also to the facilitators and policy makers 
instrumental in the training of the teachers. This feedback, coupled with 
reflective practice, at all levels of the educational system provided the 
means for teachers to improve the success of all students in their class- 
rooms. 
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two basic premises: coherence within and between the multiple 
levels of the schooling systems and a focus on evidence-informed 
inquiry into effectiveness at each level of the system. 

The first premise, coherence, was established both within and 
between levels of the education system that comprised policy 
makers, project leadership, visiting facilitators, school princi- 
pals, senior managers, and teachers. Involving whole schools in 
improving the quality of instruction, rather than focusing solely 
on individual teachers, has become increasingly common among 
initiatives to change problematic achievement profiles. Policy 
makers and researchers are beginning to recognize that teachers 
do not practice independently of the social context of their work. 
Developing within-school coherence in teaching and learn- 
ing has been a major thrust of these approaches (e.g., Madda, 
Halverson, &c Gomez, 2007; Newmann, Smith, Allensworth, 
6c Bryk, 2001). Coherence between the policy and priorities 
of district administrators and what happens at the school level 
has reduced competing, and sometimes conflicting, directives 
for schools as they grapple with change (Datnow et al., 2003). 
However, learning and change at the level of policy and project 
delivery, in response to information about project implementa- 
tion and outcomes, has not been a strong element of such ini- 
tiatives. In the project described here, policy makers and those 
responsible for project delivery provided both strong direction to 
the project and shifted their priorities in response to the result- 
ing evidence regarding implementation and outcomes. 

Coherence between research and practice, which included 
evidence-informed inquiry, was another important aspect of 
the project. Often the change is directed by, or occurs in col- 
laboration with, researchers who are responsible for articulating 
effective instructional practices, gathering data to ensure pro- 
gram fidelity, and analyzing student data to determine if what 
is enacted is having the desired effect (Gottfredson, Marciniak, 
Birdseye 6c Gottfredson, 1995; Stallings 6c Krasavage, 1986; 
Van der Sijde, 1989). But, the learning and associated change in 
these approaches is essentially the responsibility of the teachers 
and school leaders. In this project, the researchers took a differ- 
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ent role. Their reflexive research questions focused on address- 
ing issues likely to influence project outcomes with subsequent 
questions evolving in response to need. As research findings 
became available, they were discussed with the project leader- 
ship, policy makers, and, when appropriate, the professional 
development facilitators working in schools, in ways that identi- 
fied issues concerning design, understanding, and implementa- 
tion. The researchers did not monitor implementation to ensure 
program fidelity, but rather to identify what was or was not 
working. Coherence was achieved between the research agenda 
and the project as problems were discussed and resolved and new 
research and training priorities set. 

The second basic premise underpinning the evidence- 
informed inquiry approach of the project was guided by key 
principles of self-regulated learning (Butler ScWinne, 1995) at 
all levels of the system involved in creating change. Self-regulated 
learning typically is researched in relation to student learn- 
ing, and many different theoretical perspectives have emerged 
(Zimmerman, 2001). The principles used to inform action at 
the multiple levels of this project were consistent with Butler 
and Winne’s portrayal of self-regulated learners as those who 
“. . . judge performance relative to goals, generate internal 
feedback about amounts and rates of progress towards goals, 
and adjust further action based on that feedback” (p. 258). Self- 
regulation is, in their view, a deliberate, judgmental, adaptive 
process (Butler ScWinne, 1995). These notions of self-regulated 
learning were applied to multiple levels of participants including 
students, teachers, the principal and senior management, pro- 
fessional development providers, and policy makers from the 
Ministry of Education (MoE) as shown in Figure 1. Engaging 
in self-regulated learning required each to understand the proj- 
ect’s goals, and then to monitor, shape, and improve their perfor- 
mance in response to the feedback provided. 

The touchstone for judging the effectiveness of the inquiry 
process was progress in student learning, particularly in relation 
to the lowest 20% of achievers in literacy. The effectiveness of 
practice at any level of the system was judged according to the 
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Figure 1. Multiple levels of learning. 

likely impact on student learning for this population. Principles 
of effective literacy teaching practice derived from the research 
literature formed the basis of the content of the professional 
development. However, it also was accepted (and expected) that 
implementing such principles, even with high levels of fidelity, 
may not have the desired outcomes. Effective practice was not 
determined independently of the context in which it was enacted. 
Building school leaders’ and teachers’ capacity to inquire into the 
impact of their practice was a central part of the process. 

The focus on inquiry did not mean that the project was con- 
tent-free. We argue that unless participants in an inquiry process 
have sufficiently sophisticated pedagogical content knowledge 
on which to draw when difficulties with outcomes are identi- 
fied, inquiry into the effectiveness of practice is unhkely to result 
in positive change for students. Thus, the project had a strong 
emphasis on building the pedagogical content knowledge of 
school leaders and teachers. This emphasis also served to achieve 
coherence at different levels of the project. National experts in 
literacy identified effective literacy principles and practices. The 
project leadership team and others with expertise conveyed these 
principles to the facilitators, using readings, workshops, discus- 
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sions, and demonstrations. In turn, facilitators conveyed these 
key ideas to school leaders who, together with the facilitators, 
assisted teachers to understand and implement the key princi- 
ples in their classrooms. These principles and practices were not 
prescriptions for teaching. Instead, they were developed as effec- 
tive approaches to literacy teaching within a given context. 


The Educational Context 

The project involved 91 schools across New Zealand. 
Although New Zealand is a small country of approximately 4 
million people, it forms a single educational jurisdiction, and so 
it can he thought of as equivalent to a medium-sized state in the 
United States. However, one key difference is that there is no 
equivalent to district administration in New Zealand. Individual 
New Zealand schools are responsible for most operational deci- 
sions; the central Ministry of Education is responsible primarily 
for policy and funding, including national professional develop- 
ment initiatives such as the one described in this paper. 

The problem of disparities in educational outcomes, however, 
is shared with other Western educational jurisdictions. Although 
recent international studies affirm that New Zealand students 
typically perform very well in literacy, these studies have high- 
lighted disquieting trends, including a wide variation in per- 
formance and the underachievement of particular groups. New 
Zealand has one of the largest standard deviations in literacy 
achievement in the OECD (OECD, 2001; Ogle et al, 2003). 
The lowest performing 20% are 2 years or more behind the aver- 
age of their age peers. Moreover, this pattern has remained stable 
over a number of years and across different surveys (Caygill &c 
Chamberlain, 2005; Elley, 1994; Martin, MulUs, Gonzalez, & 
Kennedy, 2003; OECD, 2001, 2005). 

In 1999, a hteracy taskforce (Ministry of Education, 1999) 
established the goal of raising literacy achievement for all chil- 
dren. They advocated students getting the best possible teaching 
through high-quality teacher education to best prepare teachers. 
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strong professional leadership to support teachers, and ongo- 
ing access to professional learning and support for teachers. The 
Literacy Professional Development Project, the focus of the dis- 
cussion of this paper, was designed to reduce the persistent dis- 
parities in student outcomes. 

The purpose of this paper is to explain how the two basic 
premises of coherence within and among levels of the project 
and evidence-informed inquiry contributed to the project’s suc- 
cess. It is not intended to “gloss over” the difficulties, but rather 
to illustrate how these difficulties were identified and effective 
solutions developed. 


The Literacy Professional Development 
Project: Backgronnd Information 

The form of the project was a response by the Ministry of 
Education to a negative evaluation (Timperley, Parr, ScHigginson, 
2004) of an earlier project called Literacy Leadership. This evalu- 
ation found that a targeted focus on instructional leadership 
and evidence-informed decision making did not yield evidence 
of the desired impact on student achievement. It was appar- 
ent to the Ministry of Education officials responsible for rede- 
signing the initiative that changing student outcomes required 
a wider focus that included classroom teachers, combined with 
the support to make the intervention more intensive and firmly 
embedded in classroom practice. There also was recognition of a 
possible overestimation of the professional capacity within the 
schools’ leadership teams to engage in both evidence-informed 
decision making and to use student achievement information to 
judge the effectiveness of initiatives (Timperley et ah, 2004). The 
Literacy Professional Development Project resulted as a school- 
based, job-embedded model of professional development that 
essentially involved visiting facilitators working with individual 
schools, including the principals, nominated literacy leaders, and 
teachers. 
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Structurally, the project functioned on four levels, not unlike 
similar projects internationally At the apex was the Ministry of 
Education, whose officials contracted the professional develop- 
ment providers and monitored the progress of the project. Key per- 
sonnel from the provider’s organization combined with regional 
team leaders of school-hased facilitators, two researchers, and a 
Ministry of Education representative to constitute a leadership 
team. This team met regularly to review progress, make project 
adjustments, and plan implementation strategies. The regional 
team leaders from the leadership team coordinated and headed 
a small team of facilitators in their geographic areas who met 
together regularly. In addition, the facilitators, as a group, met 
for several days on three occasions each year at national semi- 
nars. Each facilitator worked with a small number of schools, a 
number that varied according to school size and geographic iso- 
lation. Schools appointed literacy leaders from within the staff 
to provide the main interface with the visiting faciUtator, and 
the school-based Hteracy leader was progressively up-skilled to 
become a resource and leader in an ongoing professional learn- 
ing community within each school. At other times, facilitators 
also worked directly with individual classroom teachers. 

The project providers were bound by four contracted out- 
comes determined by the Ministry of Education: evidence of 
improved student achievement, evidence of improved teacher 
content knowledge, evidence of improved transfer of under- 
standing of hteracy pedagogy to practice, and evidence of effec- 
tively led professional learning communities. In this initiative, 
professional development was designed to be based on the indi- 
vidual and collective professional learning needs of the teachers 
and leaders in each school. 

To identify these needs and, depending on the focus of the 
schools, facilitators interviewed the principal, nominated a liter- 
acy leader in the school, and observed teachers’ reading or writing 
instruction. All school personnel were asked to complete a two- 
part questionnaire. The first part involved rating a hypothetical 
classroom lesson in which the learning aims were misahgned to 
the teaching activities and feedback was provided to the students 
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using a 6-point Likert-type scale. The second part involved two 
scenarios related to use of evidence. One scenario presented a 
description of the decision-making process of a school address- 
ing a literacy problem. It was deliberately inadequate in terms 
of key aspects such as evidence-informed analysis of need, the 
match between the literacy problem and intervention measure, 
and the appropriateness of measures to test the effectiveness of 
the chosen intervention. Another scenario asked participants to 
interpret a set of student achievement data in reading or writing 
for a colleague. 

The intention was for facilitators to collate and analyze this 
teacher assessment data with the school professionals, identify- 
ing their professional learning needs. Then together they could 
construct each school’s professional development program. 
Consistent with the collaborative approach, schools were to 
ascertain their progress in achieving the contracted outcomes 
through evidence-informed discussions between the visiting 
facilitators and school leaders at 6-month intervals. These reports 
formed the basis of semiannual milestone progress reports to the 
Ministry of Education. 


Overview of Researeh 

Although the project is ongoing, the present paper relates 
to the first cohort of schools that participated in the 2004-2005 
school year. Both authors are contracted researchers on the 
project and members of the project leadership team. Thus, they 
were, at times, observers of and contributors to the research 
data and the decision-making processes that occurred in this 
forum. They observed and sometimes participated in facilita- 
tor development. Additionally, as described in the previous 
study (Parr, Timperley, Reddish, Jesson, & Adams, 2006), the 
researchers undertook independent research in a sample of 
schools using multiple methods. The data used for this paper 
represent aspects of the data from the project as a whole. 
Essentially, this research involved the analysis of the student 
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literacy outcomes in both the case study schools and all project 
schools; interviews of principals, senior managers, and teach- 
ers to understand their learning processes and the perceived 
impact of facilitators’ interventions in the case study schools; 
audiotaped feedback conversations between visiting facilitators 
and teachers following classroom observations; and written 
responses, audiotaped discussions, and interviews with visiting 
facilitators. An overview of the data collection is presented in 
Table 1 and described in greater detail below. 

Participants and Measures 

All elementary schools (N = 91) participating in the first 
cohort of the project reported their student achievement data to 
the researchers. The project involved all students from kinder- 
garten to grade 7, although achievement data represent only stu- 
dents from grade 2 onwards, as this is the earliest grade level for 
which national normative data are available. Reading achieve- 
ment data were available for 3,787 students. For writing, data 
from a sample of 1,064 students were available. The data col- 
lection for students followed a three-phase cycle: at the begin- 
ning of the project (February 2004), at the end of the first year 
(November 2004), and at the end of the second year of the proj- 
ect (November 2005). In reading, data were from a standardized 
test of reading. Supplementary Test of Reading (STAR; Elley, 
2001), alternating the use of forms A and B. In writing, data 
were obtained from a criterion-referenced (to the national cur- 
riculum) measure of writing. Assessment Tools for Teaching and 
Learning (asTTle; Glasswell, Parr, &c Aikman, 2001) that has 
associated national normative data (grades 3-7). 

Qualitative data were collected in a subset of research schools 
{n = 14), which were selected to represent a range of geographi- 
cal areas. The participants, or their caregivers, provided signed 
consent to be part of the research. All of the professional staff, 
including principals, literacy leaders, and teachers, at the reading 
schools {n = 73) and writing schools {n = 43) participated in the 
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Study. Teaching experience ranged from less than 1 year to more 
than 30 years, with a mean of approximately 14 years. 

At the beginning of the project, researchers interviewed prin- 
cipals, hteracy leaders, and 3 teachers in each school to ascertain 
their expectations of the project. They participated in a second 
interview in July 2004. At this time, they were asked to choose 
a category that described their reactions to information from 
the needs analysis process, including classroom observations, 
student achievement data, and questionnaires for the staff. The 
response category options were major new insights^ minor new 
insights^ confirmed what I knew, and not useful. As the research 
interviews proceeded, an additional category, cant remember, was 
added. At the end of each year, principals and literacy leaders 
participated in interviews to ascertain the impact of the project 
on their learning and school operations, the changing role of the 
facilitator in each school, and the process of determining their 
school’s progress on the contracted outcomes. 

Five of the facilitators provided data about facilitation practice 
in the research schools. Data collected in July 2004 ascertained 
their approach to the professional development. The interview 
questions for the facilitators paralleled those asked of the school 
personnel. Episodes of audiotaped feedback to teachers (n = 16) 
collected between June and October 2005 documented the form 
and content of the feedback of the facilitation practice. Following 
the audiotaping of facilitator feedback, interviews with teachers 
asked them what they had learned from the feedback, whether 
they agreed with the feedback, and what, if any, aspect of it they 
could or would put into practice. 

In addition, near the midpoint and at the end of the proj- 
ect, 95% of the facilitators (« = 21) responded to written ques- 
tions about the challenges they faced and their learning over 
the course of the project. At both times, following the com- 
pletion of these questionnaires, the regional teams of facilita- 
tors’ discussions were audiotaped and transcribed. A subset of 
the volunteer facilitators (n = 5) from the larger cohort was 
also interviewed to provide more in-depth understanding of 
their facilitation challenges and learning. Facilitator question - 
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naires differed according to the time of administration. In July 
2004, the questionnaire asked facilitators to provide written 
responses to a scenario that described a difficult situation in a 
school followed by a request for assistance by a senior manager 
at the school. At the end of the year, they were asked to identify 
what they had learned over the 2 years and the implications for 
further practice. 

Data Analysis 

Classroom teachers scored student reading achievement tests 
using the scoring sheets that accompanied the tests. Accuracy 
of scoring was established by sampling within each class. A 
measure of gain or progress was calculated for each school. The 
magnitude of gain over time was calculated for each class using 
Cohen’s d effect size. For writing, expert facilitators moderated 
the scoring in all schools. Because asTTle writing is scored on a 
common scale, gain was calculated from total scores at the three 
time points. 

All interviews and facilitator team discussions were coded 
according to the identified challenges and the extent to which 
the learning process and practice described were consistent with 
a co-constructed needs analysis approach. 

The protocols for the analysis of feedback episodes were 
developed from the literature on self-regulated learning (Butler 
6c Winne, 1995). Researchers coded the feedback episodes 
according to the extent to which specific goals were identified, 
the analysis of the observed practice and discussion of any rec- 
ommended changes was co-constructed, and the ways devised to 
obtain ongoing feedback were identified within the classroom 
situation. Teachers were interviewed about the usefulness of the 
feedback and their understanding of how to make recommended 
changes. The accuracy of these analyses was checked with each 
facilitator. Accuracy was not contested by any of the facilitators. 
However, in some instances, they requested that more contex- 
tual information be included when it was reported to the project 
leaders and Ministry of Education (Parr et al., 2006). 
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Results 

The project was successful in meeting the goal of raising stu- 
dent achievement in writing and reading in schools catering for 
students in kindergarten to grade 7. The average Cohen’s d effect 
size gain (Cohen, 1988) relative to where the students started on 
standardized assessments for schools that chose to focus on writ- 
ing {n = 1,064 students) was 1.28 (Ministry of Education, 2006), 
equivalent to 2.6 times the expected gain over the 2 years of the 
project. For the target group of students who scored in the low- 
est 20% in achievement, the effect size was 2.05, approximately 4 
times the expected gain over the 2 years. For reading {n = 3,787 
students), the effect size gain was 0.87, equivalent to approxi- 
mately twice the expected gain over the 2 years of the project. 
A ceiling effect on the reading assessment can partly explain the 
lower effect size, so the results for the lowest 20% of students are 
possibly more relevant. For these students, the effect size gain in 
reading was 1.97 (Ministry of Education), equivalent to approxi- 
mately 4 times the expected gain over the 2 years of the project. 
To put these gains in perspective, a recent review of the impact 
of professional development on reading and writing (Timperley, 
Wilson, Barrar, & Fung, 2007) indicates that the average proj- 
ect gains overall and the gains for the lowest achievers in this 
project are at the high end for literacy interventions, particularly 
for populations who have been traditionally underserved by the 
education system. 


Evidence-Informed Inquiry and Coherence 

The remainder of this paper seeks to explain the positive 
impact of the project. The first example of systematic inquiry 
and learning occurred when the project was formulated. The 
Ministry of Education officials became aware, as a result of 
the evaluation of the project’s predecessor, Fiteracy Feadership 
(Timperley et al., 2004), that focusing on leadership alone was 
insufficient to change teaching practice in ways that improved 
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student outcomes. Leaders were not sufficiently focused on stu- 
dent outcomes and typically did not have the literacy knowledge 
to work in sufficient depth with their teachers. Hence, the new 
project involved facihtators with expertise in literacy working 
with both teachers and leaders. 

As a result of the previous project, the Ministry of Education 
also reahzed the need for clarity in contracting the outcomes, 
including improved student achievement and directing teacher 
learning toward this end. In the new project, the contracted out- 
comes formed a key role in achieving project coherence. The 
project leaders accepted these as relevant and important foci 
and structured the project to provide the conditions necessary 
to achieve them. The team of facilitators understood the target 
outcomes; their semiannual milestone reports addressed each of 
them. The reporting of the project leaders to the Ministry of 
Education also included progress updates to each of the target 
outcomes. Moreover, the milestone format provided a frame- 
work that assisted facilitators and schools to inquire in evidence- 
informed ways into their progress in meeting the contracted 
outcomes. Progressions toward achieving each were identified 
with criterion statements to enable schools and facilitators to 
judge progress and identify the evidence to substantiate a par- 
ticular judgment. 

For example, in the progressions that referred to building 
teachers’ content knowledge and the transfer of that knowledge 
into practice, schools were considered to be in the initial stage 
when they provided evidence that they were constructing an 
informed knowledge base. Progressions in this domain included 
building knowledge and implementing change. The final phase 
was evaluating and sustaining change. Determining from the evi- 
dence the extent of progress toward each outcome was coopera- 
tive, evidence-informed, and, deliberately, a learning process for 
schools. Facihtators worked with school leaders in each of their 
schools to draw inferences from the collected evidence at mutu- 
ally agreed upon points in time. School principals and senior 
managers had typically worked in an input-focused environment 
and believed that the project involved simply updating teach- 
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ers about effective literacy practices. Through this process, they 
gradually came to realize that, for the project to be successful, 
their school needed to adopt a whole-school improvement focus 
to achieve coherence in both practice and the inquiry processes 
within the schools. Leadership knowledge and skills became an 
increasing focus within the project. 

The emphasis on developing evidence-informed inquiry took 
a different form at each level of the project. In the first year of the 
project, the focus was on teachers and their students. Participants 
at other levels, including their leaders, were not charged with the 
same extent of learning and change. The focus for the teachers 
was to develop two interrelated areas of knowledge as a basis 
for their learning. The first area was to deepen their pedagogical 
content knowledge in fostering reading comprehension or writ- 
ing, and the second was to use a variety of evidence of student 
learning to monitor whether their existing or changed strategies 
were having the desired effect on student learning. Personal pro- 
fessional learning goals were developed with each participating 
teacher in relation to these areas. 

At the student level, the focus was on assisting students to 
understand explicit learning goals, as opposed to performance 
goals (Dweck, 1999) for particular activities, articulating criteria 
for success, and developing self- and peer systems for monitor- 
ing their progress toward their learning goals. 

Although facilitators accepted they should improve their lit- 
eracy content knowledge, the need to set personal, specific learn- 
ing goals; to receive feedback; and to monitor progress toward 
these goals was not an initial focus. However, as the project 
evolved, it became increasingly apparent through observations 
and interviews in the schools that facilitators should develop 
learning at deeper and more personal levels to work effectively 
with the diversity of teachers. More details of how this occurred 
are described in the example in the following section. 

Similarly, the members of the leadership team responsible 
for improving facilitator knowledge became aware that they also 
needed to have their practice open to similar levels of scrutiny. 
Learning needed to extend beyond teachers and their leaders 
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to include those responsible for delivery of professional devel- 
opment. As a result, in the second year, they also developed 
learning goals and gathered evidence about their effectiveness 
in achieving these goals. Policy makers and project leaders also 
realized project facilitation required greater emphasis, and they 
agreed on a new contracted outcome, namely evidence of effec- 
tive facilitator practice. Effective project leadership practice was 
part of this outcome. 

Goals and the monitoring of progress and feedback, the 
essence of self-regulatory learning (Butler & Winne, 1995), 
became an integral part of the learning process at each level of 
the project. Operating iteratively and recursively, this cycle of 
evidence gathering, evaluation against goals, and adjustment of 
practice comprised a system within which learning could take 
place at all levels. When analyzing the project’s learning pro- 
cesses, we invoke a broader systems perspective of self-regulation 
to think about the project itself as a learning project; activities at 
one system level influenced and enabled learning at another. 


Theory in Practice: An Example 
of How the Inqniry Process Promoted 
Learning and Coherence at All Levels 

The above, relatively abstract description of how the proj- 
ect functioned as an inquiry-based project, is exemplified in this 
section by describing how evidence from one level in a specific 
situation resulted in changes and outcomes at other levels, lead- 
ing to greater project effectiveness and coherence. This example 
is one of many. 

Interactions at all levels of the project centered on evidence. 
This evidence could include evidence of student learning from 
assessments and interviews, evidence of teacher practice from 
observations and their knowledge from their written responses 
to a hypothetical lesson scenario, evidence of school leadership 
practice from observations of meetings, or evidence of facilitator 
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practice from feedback sessions and teacher responses. Evidence 
was used to hone practice to achieve outcomes. 

The evidence served two important functions. The first was 
to identify learning needs of participants at the different levels, 
and the second was to determine the extent to which those learn- 
ing needs had been met. In keeping with these functions, initial 
work by facilitators in each school focused on obtaining evidence 
to undertake an analysis of both student and professional learn- 
ing needs. As the project developed, professional meetings were 
observed to identify leadership needs in this area. These data 
were intended to be used to plan professional learning. 

However, it was clear from responses to the interviews with 
school-based personnel 6 months into the project that few prin- 
cipals, literacy leaders, or teachers perceived the reason for this 
needs analysis, particularly as it might concern them. Thirty 
percent of the teachers interviewed did not remember receiv- 
ing feedback from the observations, nor did they understand 
how their responses to the scenario had been analyzed or used. 
The remainder indicated that the process had led to minor new 
insights or confirmed what they already knew. Facilitator inter- 
views established that they were insufficiently clear themselves, 
at this early point in the project’s history, to explain the purpose 
of the needs analysis to the school-based professionals and, more 
importantly, to illustrate how the data would inform their activi- 
ties within the schools. Initially, both schools and most facilita- 
tors viewed the needs analysis as an exercise to get through as 
expeditiously as possible. Although they were familiar with the 
diagnostic use of data in the classroom, they were not familiar 
with utilizing schoolwide data in ways that might inform whole- 
school action. Schools and facilitators steeped in a traditional 
delivery model of professional development were anxious to start 
the professional development delivery sessions. 

Consistent with this view, it also was established that the 
semiannual milestone reports were frequently being completed 
by facilitators alone who perceived it to be their job to report on 
schools, rather than to discuss the progressions with school lead- 
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ers. The participants were missing an important co-constructed 
learning opportunity. 

These findings from the research interviews were shared and 
considered at a project leadership meeting. The leadership team 
realized that the planned needs analysis process required that 
facilitators first recognize the concept of a needs analysis and 
the advantages of a more co-constructed approach. Further, they 
realized that facilitators needed to possess considerable skill to 
utilize the data from each school to challenge current practices. 
The project leadership team considered the issue as one of under- 
standing the need for, and supporting the acquisition of, deeper 
knowledge of facilitators with regard to data interpretation. Also, 
the facilitators needed more expertise in how to use this data to 
change teacher and leader practice. This realization led to the 
identification of five core acts of facilitation and the subsequent 
training of facilitators. The first of these acts stated that all inter- 
actions and activities should be contracted to develop shared 
understanding of the reasons for the activity, shared agreement 
about its relevance, and shared responsibility for presenting the 
evidence and for formulating the next steps. The second core act 
specified that every piece of evidence should be discussed with 
those involved as it was collected. The third act followed from 
the first two by specifying that action plans should be based on 
the facilitators’, teachers’, and school leaders’ shared understand- 
ing of the evidence related to professional and student learning 
needs at the time. The fourth provided a focus for change by 
specifying that all activities should be focused on the question, 
“What needs to change to improve?” Particular reference was 
made to feedback that not only affirmed teacher practice but also 
challenged that practice. The final act related to the leadership of 
schools and stated that challenge also may involve challenging 
management practices when this proved necessary (Learning 
Media, 2004). With guidance from the project leadership team, 
at national seminars and in regional teams, facilitators built their 
knowledge and skills in these acts of facilitation. 

When school leaders worked with facilitators to use student 
data to identify student learning needs, they also began to see the 


1 08 Journal of Advanced Academics 


Timperley & Parr 


relevance of examining what teachers knew or needed to know 
in order to meet these needs. Combining student data with 
classroom observations and data about teacher knowledge from 
responses to the scenarios gave facilitators and teachers evidence 
from which to draw inferences about practice and how it needed 
to improve. This evidence became the basis for conversations 
that included the joint analysis and subsequent co-construction 
of new practice. 

The leadership team supported facilitator learning about how 
to undertake such conversations. The leadership team provided 
multiple opportunities to learn and high accountability require- 
ments. To approach their work in more co-constructed ways, 
facilitators required knowledge of how to construct interactions 
that enabled the conditions necessary to work effectively in a 
school, how to learn from evidence, and how to give construc- 
tive, useful feedback to teachers, literacy leaders, and principals. 

Most facilitators were new to the role. They did not under- 
stand the notion of co-constructed conversations around evidence 
or the notion of catering to the diversity of teacher learners. They 
were provided with some direct instruction, but mainly, they 
were given opportunities to observe, discuss, and participate, 
and analyze their own practice. Responses to research tasks and 
interviews in the second year identified that this training was the 
area of greatest learning. The project leadership team came to 
appreciate the complexity of what was involved as they engaged 
in the process of analyzing their own practice. A focal point of 
the research became evidence of more effective facilitator prac- 
tice. This occurred through an in-depth analysis of episodes 
of teacher learning from feedback. A research analysis of the 
sample of the facilitator’s feedback showed that initial feedback 
was often generalized, with criticisms embedded in manipula- 
tive questioning. Facilitators used this technique in the hope that 
teachers would say what needed to improve, thus providing an 
opportunity for facihtators to tell the teachers how they should 
change. Following discussion about these findings from the 
research, each facilitator then audiotaped his or her own feed- 
back and analyzed it according to the protocols of conversations 
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consistent with the joint analysis of existing practice and co- 
construction of new practice. Then the facilitators discussed with 
colleagues how they might change. This ever-increasing scrutiny 
of practice was initially uncomfortable; however, participants 
gradually grew to appreciate its value, as revealed in subsequent 
interviews of facilitators. 

The new approaches adopted by the facilitators helped school 
leaders to understand that the project required comprehensive 
school improvement, not just updating their teachers about lit- 
eracy teaching and learning. This meant that school principals 
could no longer stay on the sidelines; they needed to become 
involved in developing evidence-informed leadership practices 
and to shift their roles toward a greater focus on instructional 
leadership. A principals’ group was established to advise the proj- 
ect leaders, in response to these new roles. 

Part of the project’s learning and response to the identified 
difficulties also occurred at the level of policy. It became increas- 
ingly apparent that the learning needed could not be undertaken 
in one year, but a second year was needed in order to develop and 
consolidate the required skills and knowledge. The Ministry of 
Education officials advocated for and obtained a second year of 
funding for each school. A second cohort of schools is now in 
the project, but the ongoing learning and adjustment within and 
among the levels is continuing. 


Discussion 

Previous attempts to change entrenched profiles of student 
achievement have been met, in the most part, with hmited suc- 
cess, particularly in projects of a similar scale. The research litera- 
ture documents and explains the problem extensively. The more 
simplistic explanations blame the policy makers for being unre- 
alistic, unclear about their goals, or having low credibility with 
politically suspect reasons (e.g., Hargreaves, 2002; Mintrop,2002). 
Alternatively, other explanations blame the attitudes, skills, and 
knowledge of practitioners responsible for implementation (e.g.. 
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Delpit, 1995; McLaughlin ScTalbert, 1993). More sophisticated 
analyses have allowed researchers to recognize that implementa- 
tion failures often occur as a result of the complex interactions 
between the proposed change, as advocated by policy makers, 
and the existing norms, belief systems, and practices of those 
responsible for implementation (Coburn, 2001; Spillane, Reiser, 
6c Reimer, 2002). This position recognizes that many of the dif- 
ficulties occur at the interface between the changes proposed by 
the initiators and those responsible for implementation. 

The project described in this paper deUberately created struc- 
tures involving overlapping levels of personnel encompassing 
policy makers and those responsible for dehvering the project to 
schools and the practitioners, thus reducing potential misunder- 
standings at the interface. These permeable layers provided the 
opportunity to develop systems for learning and project devel- 
opment. The learning settings were structured to support project 
implementation, but both these settings and their foci changed 
to support project learning and improvement. Over time, the 
expectation developed that everyone needed to learn, not just 
the teachers and their leaders. The pivotal role of the facilitators 
to deliver these project messages to schools and develop their 
capacity to respond became increasingly apparent. Problems of 
implementation were seen as opportunities for those involved to 
inquire and engage in a problem-solving process. If one group did 
not have the capacity to implement particular practices, rather 
than attribute blame, learning opportunities were created. 

Providing consistently focused learning opportunities 
throughout the levels of the project was instrumental in creating 
project coherence. Through the commitment to learn from evi- 
dence, misunderstandings were identified and greater coherence 
at the level of the principles underpinned the project. Coherence 
was not established through more detailed implementation man- 
uals, but through a deeper understanding of the project approach 
and the knowledge, skills, and rationale to enact those principles. 

The developing realization that it was not just schools, their 
leaders, and teachers who needed to learn and change, but that 
the visiting facilitators, project leaders, and policy makers also 
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needed to learn and change is not well documented in the lit- 
erature. A search of the literature focusing on effective facilita- 
tion practices resulting in the kinds of teacher change that has 
a substantive impact on student learning revealed few empirical 
studies (Timperley et ah, 2007). If we are to solve entrenched 
student achievement problems through teacher learning, it may 
be time to turn our focus to the skills and knowledge required 
for those providing the professional learning opportunities. In 
doing so, our focus on promoting student and teacher learning 
must remain the goal. Ultimately, it is within classrooms and 
student- teacher interactions where activities of highest leverage 
occur. However, if we are to improve the learning opportunities 
for students, then creating learning opportunities and expecting 
changes throughout the system needs to become a necessary and 
integral part of the process. 
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